Robert F. Kennedy Jr. is an AI guy. Last week, during a stop in Nashville on his Take Back Your Health tour, the Health and Human Services secretary brought up the technology between condemning ultra-processed foods and urging Americans to eat protein. "My agency is now leading the federal government in driving AI into all of our activities," he declared. An army of bots, Kennedy said, will transform medicine, eliminate fraud, and put a virtual doctor in everyone's pocket. RFK Jr. has talked up the promise of infusing his department with AI for months.

AI models are increasingly prevalent in high-stakes environments, necessitating1 thorough assessment oftheir capabilities and risks.

From jogging and cycling to multi-tasking or puttering around the house, open earbuds are an excellent way to jam out in the real world. If you've done any wireless earbuds shopping lately, you've likely noticed a new design category cropping up everywhere. They're called open earbuds (or open-ear buds, depending on the brand), and just about every audio brand has a pair (or three). They come in a slew of styles, but most either loop around your ears like older Beats buds, or clip on like funky-futuristic earrings. Whatever the style, they're designed to deliver satisfying sound while keeping your ear canals open to the sounds of the world around you.

How to identify the first iconic use case for an enterprise AI transformation. Many organizations rushed into generative AI, only to see pilots fail to deliver value . Now, companies want measurable outcomes--but how do you design for success? At Mistral AI, we partner with global industry leaders to co-design tailored AI solutions that solve their most difficult problems. Whether it's increasing CX productivity with Cisco, building a more intelligent car with Stellantis, or accelerating product innovation with ASML, we start with open frontier models and customize AI systems to deliver impact for each company's unique challenges and goals. Our methodology starts by identifying an iconic use case, the foundation for AI transformation that sets the blueprint for future AI solutions.

To mitigate these risks, current evaluation benchmarks predominantly employ expert-designed contextual scenarios to assess how well LLMs align with human values. However, the labor-intensive nature of these benchmarks limits their test scope, hindering their ability to generalize to the extensive variety of open-world use cases and identify rare but crucial long-tail risks. Additionally, these static tests fail to adapt to the rapid evolution of LLMs, making it hard to evaluate timely alignment issues. To address these challenges, we propose ALI-Agent, an evaluation framework that leverages the autonomous abilities of LLM-powered agents to conduct in-depth and adaptive alignment assessments. ALI-Agent operates through two principal stages: Emulation and Refinement.

Multimodal foundation models that can holistically process text alongside images, video, audio, and other sensory modalities are increasingly used in a variety of real-world applications. However, it is challenging to characterize and study progress in multimodal foundation models, given the range of possible modeling decisions, tasks, and domains. In this paper, we introduce Holistic Evaluation of Multimodal Models (HEMM) to systematically evaluate the capabilities of multimodal foundation models across a set of 3 dimensions: basic skills, information flow, and real-world use cases. Basic multimodal skills are internal abilities required to solve problems, such as learning interactions across modalities, fine-grained alignment, multi-step reasoning, and the ability to handle external knowledge.